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ABSTRACT 



In 1990, the National Assessment of Educational Progress 
(NAEP) included a Trial State Assessment (TSA) ; for the first time in NAEP ' s 
history, voluntary state-by-state assessments were made. In 1992, the NAEP 
Program included an expanded TSA in fourth-grade reading. The 1992 reading 
assessment considered students' performance in situations that involved 
reading different kinds of materials for different purposes. The fourth-grade 
assessment measured two global purposes for reading- -reading for literary 
experience and reading to gain information. In Michigan, 2,446 students in 
100 public schools were assessed. This report describes the reading 
proficiency of Michigan fourth-graders and compares their overall performance 
to students in the Central region of the United States and the nation (using 
data from the NAEP national assessments) . The distribution of reading results 
and reading achievement level results are provided for subpopulations of 
students (race/ethnicity, type of location, parents' educational level, and 
gender, and performance according to purpose for reading) . To provide a 
context for the assessment data, participating public school students, their 
reading teachers, and principals completed questionnaires which focused on: 
policies and practices related to reading (time for instructional activities 
and instructional resources for reading) ; delivery of reading instruction 
(instructional materials and activities, workbooks/worksheets/writing, 
discussion and group activities, time to read, reading and use of libraries, 
assessing progress in reading) ; reading instructors (preparation, experience 
and professional development); students' home support for literacy (reading 
outside of school and in the home and hours of television watched per day) . 



The average reading proficiency of fourth-grade public school students in 
Michigan on the NAEP reading 1« was '117 compared to 216 nationwide. (SR) 



er!c 




NATIONAL CENTER FOR EDUCATION STATISTICS 



NAEP 1992 

Reading State Report for 
Michigan 

The Trial State Assessment at Grade 4 



4 



A 



U S DEPARTMENT OF EDUCATION 
Otfice of Educational Research and Improvement 

educational resources information 
'/ CENTER (ERIC) 

(7/This document has been reproduced as 
T received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



to 



to 



to 

to 



Jl 



I r? 



77 *. 















- ERLC l 

^ I 



THE NATION’S 

PORT |rcaep 
:ard 



Prepared by Educational Testing Service under contract 
with the National Center lor Education Statistics 

Office of Educational Research and Improvement 
U.S. Department of Education 







What is The Natiom’s Report Card ? 



THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally representative and continuing assessment 
of what America’s students know and can do in various subject areas. Since 1969, assessments have been conducted periodically in reading, mathematics, 
science, writing, history/geography, and other fields. By making objective information on student performance available to policymakers at the national, 
state, and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only information related to academic 
achievement is collected under this program. NAEP guarantees the privacy of individual students and their families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of Education. The Commissioner of 
Education Statistics is responsible, by law, for carrying out the NAEP project through competitive awards to qualified organizations. NAEP reports directly 
to the Commissioner, who is also responsible for providing continuing reviews, including validation studies and solicitation of public comment, on NAEP’s 
conduct and usefulness. 

In 1988, Congress created the National Assessment Governing Board (NAGB) to formulate policy guidelines for NAEP. The board is responsible for 
selecting the subject areas to be assessed, which may include adding to those specified by Congress; identifying appropriate achievement goals for each age 
and grade; developing assessment objectives; developing test specifications; designing the assessment methodology; developing guidelines and standards 
for data analysis and for reporting and disseminating results; developing standards and procedures for interstate, regional, and national comparisons; improving 
the form and use of the National Assessment; and ensuring that all items selected for use in the National Assessment are free from racial, cultural, gender, 
or regional bias. 

The National Assessment Goverminig Board 



Mark D. Musick, Chairman 
President 

Southern Regional Education Board 
Atlanta, Georgia 

Hon. William T. Randall, Vice Chair 
Commissioner of Education 
State Department of Education 
Denver, Colorado 

Parris C. Battle 
Education Specialist 
Dade County Public Schools 
Miami, Florida 

Honorable Evan Bayh 
Governor of Indiana 
Indianapolis, Indiana 

Mary R. Blanton 
Attorney 

Blanton & Blanton 
Salisbury, North Carolina 

Boyd W. Boehlje 

Attorney and School Board Member 
Pella, Iowa 

Linda R. Bryant 
Dean of Students 

Florence Reizenstein Middle School 
Pittsburgh, Pennsylvania 

Naomi K. Cohen 

Office of Policy and Management 

State of Connecticut 

Hartford, Connecticut 

Charlotte Crabtree 
Professor 

University of California 
Los Angeles, California 



Chester E. Finn, Jr. 

Founding Partner and Senior Scholar 
The Edison Project 
Washington, DC 

Michael S. Glode 

Wyoming State Board of Education 

Saratoga, Wyoming 

William Hume 
Chairman of the Board 
Basic American, Inc. 

San Francisco, California 

Christine Johnson 
Director of Urban Initiatives 
Education Commission of the States 
Denver, Colorado 

John S. Lindley 
Principal 

Galloway Elementary School 
Henderson, Nevada 

Honorable Stephen E. Merrill 
Governor of New Hampshire 
Concord, New Hampshire 

Jason Millman 
Professor 

Cornell University 
Ithaca, New York 

Honorable Richard P. Mills 
Commissioner of Education 
State Department of Education 
Montpelier, Vermont 

Carl J. Moser 
Director of Schools 

The Lutheran Church — Missouri Synod 
St. Louis, Missouri 



John A. Murphy 
Superintendent of Schools 
Charlotte-Mecklenburg Schools 
Charlotte, North Carolina 

Michael T. Nettles 
Professor 

University of Michigan 
Ann Arbor, Michigan 

Honorable Carolyn Pollan 
Arkansas House of Representatives 
Fort Smith, Arkansas 

Thomas Topuzes 
Senior Vice President 
Valley Independent Bank 
El Centro, California 

Marilyn Whirry 
English Teacher 
Mira Costa High School 
Manhattan Beach, California 

Sharon P. Robinson 
Assistant Secretary for Educational 
Research and Improvement (Ex-Officio) 
U.S. Department of Education 
Washington, D.C. 



Roy Truby 

Executive Director, NAGB 
Washington, D.C. 




3 



NATIONAL CENTER FOR EDUCATION STATISTICS 



NAEP 1992 

Reading State Report for 
Michigan 

The Trial State Assessment at Grade 4 




Report No. 23-ST07 September 1993 




THE NATION’S 
REPORT 
CARD 




Prepared by Educational Testing Service under contract 
with the National Center for Education Statistics 

Office of Educational Research and Improvement 
U.S. Department of Education 

4 



U.S. Department of Education 

Richard W. Riley 

Secretary 

Office of Educational Research and Improvement 
Sharon P. Robinson 
Assistant Secretary 

National Center for Education Statistics 

Emerson J. Elliott 

Commissioner 



FOR MORE INFORMATION: 

For ordering information on this report, write: 

Education Information Branch 

Office of Educational Research and Improvement 

U.S. Department of Education 

555 New Jersey Avenue, NW 

Washington, D.C. 20208-5641 

or call 1-800-424-1616 (in the Washington, D.C. metropolitan area call 202-219-1651). 

Library of Congress, Catalog Card Number 93-79252 
ISBN: 0-88685-148-3 

The work upon which this publication is based was performed for the National Center for Education Statistics, Office of Educational Research 
and Improvement, by Educational Testing Service. 

Educational Testing Service is an equal opportunity, affirmative action employer. 

Educational Testing Service, ETS, and the ETS logo are registered trademarks of Educational Testing Service. 




5 



Table of Contents 

INTRODUCTION 1 

EXECUTIVE SUMMARY 5 

OVERVIEW 15 

This Report 17 

Guidelines for Analysis 20 

Profile of Michigan 21 

Fourth-Grade School and Student Characteristics ! 21 

Schools and Students Assessed 22 

PART ONE 

How Proficient in Reading are Fourth-Grade Students in 

Michigan Public Schools? 25 

Chapter 1. Students’ Reading Performance 27 

Levels of Reading Achievement 30 

Describing Fourth-Grade Students’ Performance at the Achievement Levels ...34 
Performance According to Purpose for Reading 40 

Chapter 2. Reading Performance by Subpopulations 43 

Race/Ethnicity 43 

Type of Community 48 

Parents’ Education Level 51 

Gender 56 

Performance According to Purpose for Reading 57 



6 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



PART TWO 

Finding a Context for Understanding Students’ Reading Proficiency ...61 

Chapter 3. Policies and Practices Related to Reading 63 

Time for Instructional Activities 64 

Instructional Emphasis 66 

Summary 68 

Chapter 4. How Is Reading Instruction Delivered? 69 

Instructional Materials for Reading 69 

Instructional Resources for Reading 71 

Emphasis on Aspects of Reading 74 

Instructional Activities 76 

Workbooks, Worksheets, and Writing 76 

Discussion and Group Activities 79 

Time to Read 81 

Reading and Use of Libraries 82 

Assessing Progress in Reading 85 

Summary 87 

Chapter 5. Who Is Teaching Reading to Fourth Graders? 89 

Preparation and Experience 89 

Professional Development 91 

Summary 93 

Chapter 6. Students’ Home Support for Literacy 95 

Reading Outside of School 95 

Reading in the Home 99 

Hours of Television Watched Per Day 102 

Summary 103 

APPENDIX A. Procedural Appendix 105 

APPENDIX B. Reading Stimuli and Example Items 117 

APPENDIX C. Setting the Achievement Levels 125 

APPENDIX D. Anchoring the Achievement Levels 129 

APPENDIX E. Data Appendix 135 

7 



ii 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



List of Tables 



Table 1. Profile of Fourth-Grade Public-School Students in Michigan, 

the Central Region, and the Nation 22 

Table 2. Profile of the Fourth- Grade Population Assessed in Michigan 24 

Table 3. Percentiles of Reading Proficiency for Fourth-Grade Public-School Students .29 
Table 4. Percentiles of Reading Proficiency for Fourth-Grade Public-School 

Students by Purpose for Reading 41 

Table 5. Percentiles of Reading Proficiency for Fourth-Grade Public-School 

Students by Race/Ethnicity 45 

Table 6. Percentiles of Reading Proficiency for Fourth-Grade Public-School 

Students by Type of Community 49 

Table 7. Percentiles of Reading Proficiency for Fourth-Grade Public-School 

Students by Parents’ Level of Education 53 

Table 8. Percentiles of Reading Proficiency for Fourth-Grade Public-School 

Students by Gender 57 

Table 9. Fourth-Grade Public-School Students’ Average Reading Proficiency 

for “Purpose for Reading” by Subpopulation 59 

Table 10. Reading Policies and Practices in Michigan Fourth-Grade 

Public Schools 64 

Table 11. Teachers’ Reports on Time Spent Teaching Reading 65 

Table 12. Teachers’ Reports on Emphasis Given to Specific Methods for 

Teaching Reading 67 

Table 13. Teachers’ Reports on Instructional Materials for Reading 70 

Table 14. Teachers’ Reports on the Availability of Resources 71 

Table 15. Teachers’ Reports on Resources for Reading Instruction 73 

Table 16. Teachers’ Reports on Emphasis on Aspects of Reading 75 

Table 17. Teachers’ and Students’ Reports on Workbooks, Worksheets, and Writing ....78 
Table 18. Teachers’ and Students’ Reports on the Frequency of Discussion 

and Group Activities 80 

Table 19. Teachers’ and Students’ Reports on the Frequency of Reading in Class 82 

Table 20. Teachers’ Reports on Sending Students to the Library 83 

Table 21. Teachers’ Reports on Assigning Books from the Library 84 

Table 22. Teachers’ Reports on Assessing Progress in Reading 86 

Table 23. Teachers’ Reports on Their Fields of Study and Teaching Experience 90 

Table 24. Teachers’ Reports on Their In-Service Education 91 

Table 25. Teachers’ Reports on Training in Specific Reading Areas 92 

Table 26. Students’ Reports on Reading for Fun 96 

Table 27. Students’ Reports on the Number of Books Read Outside of School 

in the Past Month 97 

Table 28. Students’ Reports on Taking Books Out of the Library 99 



^ ^ ~ 8 

THE 1992 NAEP TRIAL STATE ASSESSMENT 



Table 29. Students’ Reports on Types of Reading Materials in the Home 100 

Table 30. Students’ Reports on Talking With Friends and Family About Reading ...101 

Table 31. Students’ Reports on the Amount of Time Spent Watching Television 

Each Day 103 

Table Al. Student Score-Level Percentages for Extended Constructed- Response 

Example Items at Grade 4 117 

List of Figures 

Figure 1. Regions of the Country 20 

Figure 2. Fourth-Grade Public-School Students’ Average Reading Proficiency 29 

Figure 3. Levels of Reading Achievement at Grade 4 31 

Figure 4. Levels of Fourth-Grade Public-School Students’ Reading Achievement 38 

Figure 5. Fourth-Grade Public-School Students’ Average Reading Proficiency 

According to Purpose for Reading 40 

Figure 6. Fourth-Grade Public-School Students’ Average Reading Proficiency 

by Race/Ethnicity 44 

Figure 7. Levels of Fourth-Grade Public-School Students’ Reading Achievement 

by Race/Ethnicity 46 

Figure 8. Fourth-Grade Public-School Students’ Average Reading Proficiency by 

Type of Community 48 

Figure 9. Levels of Fourth-Grade Public-School Students’ Reading Achievement 

by Type of Community 50 

Figure 10. Fourth-Grade Public-School Students’ Average Reading Proficiency by 

Parents’ Level of Education 52 

Figure 11. Levels of Fourth-Grade Public-School Students’ Reading Achievement 

by Parents’ Level of Education 54 

Figure 12. Fourth-Grade Public-School Students’ Average Reading Proficiency 

by Gender 56 

Figure 13. Levels of Fourth-Grade Public-School Students’ Reading Achievement 

by Gender 58 

Figure Al. Description of Reading Purposes 107 

Figure A2. Description of Reading Stances 108 

Figure A3. Cutpoints for Achievement Levels at Grade 4 126 

Figure A4. Anchor Descriptions of Achievement Levels 132 

9 

iv THE 1992 NAEP TRIAL STATE ASSESSMENT 



Michigan 



INTRODUCTION 



THE NATION’S 
REPORT 
CARD 



1992 



rsnp 

m 



Trial State Aeeeeament 



The National Assessment of Educational Progress (NAEP) is a Congressionally mandated 
project of the National Center for Education Statistics (NCES) that has collected and 
reported information for nearly 25 years on what American students know and what they 
can do. It is the nation's only ongoing, comparable, and representative assessment of 
student achievement. Its assessments are given to scientifically selected samples of youths 
attending both public and private schools and enrolled in grades four, eight, or twelve. The 
assessment questions are written around a framework prepared for each content area — 
reading, writing, mathematics, science, and others — that represents the consensus of groups 
of curriculum experts, educators, members of the general public, and user groups on what 
should be covered on such an assessment. Reporting includes means and distributions of 
scores, as well as more descriptive information about the meaning of the data. 



New Reading Assessment Framework and Questions 



The goal of the National Center for Education Statistics is to make data available for the 
public and to do so in accurate and understandable ways that are not misleading. The task 
is challenging because much of what matters in NAEP is changing: 



• the content in response to the developing standards of various curricular 
groups; 

• the assessment questions in response to new developments in assessments; 
and 

• the reporting in response to increasing interest in student achievement 
relative to standards of student performance. 
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Michigan 



The framework for the 1992 Trial State Assessment Program in reading considered 
students’ performance in situations that involved reading different kinds of materials for 
different purposes. The fourth-grade reading assessment measured two global purposes for 
reading — reading for literary experience and reading to gain information. (The eighth- and 
twelfth-grade national NAEP reading assessments also measured a third purpose for 
reading — reading to perform a task.) Reading for literary experience usually involves the 
reading of novels, short stories, plays, and essays. In these reading situations, the reader 
can determine how the author explores or uncovers experiences through the text and 
considers the interplay among events, emotions, and possibilities. Reading to gain 
information usually involves the reading of articles in magazines and newspapers, chapters 
in a textbook, entries in encyclopedias and catalogs, and entire books on particular topics. 
These reading situations call for different orientations to text from those in reading for 
literary experience because readers are specifically focused on acquiring information. 

The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 

• Initial Understanding — comprehending the overall or general meaning of 
the selection. 

• Developing an Interpretation -- extending the ideas in the text by making 
inferences and connections. 

• Personal Response - making explicit connections between ideas in the text 
and a student’s own background knowledge and experiences. 

• Critical Stance — considering how the author crafted a text. 

These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. 

The 1992 NAEP reading assessment uses a variety of innovative assessment approaches 
that are considered significant advancements over previous assessments. In addition to 
multiple-choice questions, the assessment primarily includes constructed-response 
.questions that ask students to demonstrate comprehension beyond a surface level. Also, 
longer and naturally-occurring reading materials are used to provide more realistic reading 
experiences than in previous assessments. 

Taken together, the changes in the 1992 reading framework and assessment activities 
preclude any comparisons between the results in this report and those for previous NAEP 
reading assessments. If the current NAEP framework is used in the future, as planned in 
the 1994 assessment, the 1992 reading data will supply the basis for a trend report. 
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A Transition in Reporting 

Over time there have been many changes in emphasis of NAEP reporting, both to take 
advantage of new technologies and to reflect changing trends in education. In 1984, a new 
technology called Item Response Theory (IRT) made it possible to create “scale scores” 
for NAEP similar to those the public was accustomed to seeing for the annual Scholastic 
Aptitude Test (SAT). Educational Testing Service, in its role as Government grantee 
carrying out NAEP operations, devised a new way to describe performance against this 
scale, called “anchor levels.” Starting in 1984, NAEP results were reported by “anchor 
levels.” Anchor levels describe distributions of performance at selected points along the 
NAEP scale (i.e., standard deviation units). Anchor levels show how groups of students 
perform relative to each other, but not whether this performance is adequate. 

In 1988, Congress established the National Assessment Governing Board (NAGB), 
assigning it broad policy making authority over NAEP, including the authority to take 
“appropriate actions ... to improve the form and use of the National Assessment” and to 
identify “appropriate achievement goals for each . . . grade and subject area to be tested in 
the National Assessment.” To carry out its responsibilities, NAGB developed 
“achievement levels,” which are collective judgments about how students should perform 
relative to a body of content reflected in the NAEP frameworks. The result is translated 
onto ranges along the NAEP scale. For the 1992 reading assessment, this process was 
conducted for NAGB under contract by American College Testing (ACT), which has 
extensive experience in standard-setting in many fields. 

With this background, the initial reports for the 1992 reading assessment mark NCES’s 
continued attempt to shift to standards-based reporting of National Assessment statistics. 
The first transition to reporting NAEP results by achievement levels was for the NAEP 
1992 Trial State Assessment in mathematics. 1 The impetus for this transition lies in the 
belief that NAEP data will take on more meaning for the public if they show what 
proportion of our youth are able to meet judgmental standards of performance. 

Reporting NAEP results on the basis of achievement levels represents a significant change 
in practice for NCES. On occasion, this agency makes use of emerging analytical 
approaches that permit new, and sometimes controversial, analyses to be done. When 
doing so, this agency, just as other statistical agencies do when introducing new measures 
to supplement or replace old measures, also has provided the data according to the earlier 
procedures in addition to the new ones. In the case of the 1992 mathematics assessment, 
for example, the “anchor levels” or “scale anchoring” method of reporting was presented 
in an appendix. 



1 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993) and the individual 
1992 Mathematics State Reports. 
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